Info-Mac 4/Info_Mac IV CD-ROM (Pacific HiTech Inc.)(August 1994).iso/Science/Propagator 1.0 Demo/Manual

Text File | 1994-02-09 | 21KB | 465 lines

               Propagator for Macintosh Demo User's Manual
                         February 9, 1994



**** Table of Contents

     Section 1 - Introduction
     Section 2 - Using Propagator Demo
         Running an Example Problem
     Section 3 - Preparing Your Own Data Files
     Section 4 - The Control Window and the Dialog Boxes
          Control Window
          Architecture Dialog
          Training Dialog
          Testing Dialog
     Section 5 - The Graphs
     Section 6 - Other Features
     Section 7 - Ordering Information



**** Section 1 - Introduction

Propagator is a commercial neural network development product produced by ARD
Corporation.  Propagator runs several variations of the most common neural
network paradigm, backpropagation.  There are versions of Propagator that run
on the Windows-based IBM PC, Macintosh, and Sun Workstations.  This demo
version is for the Macintosh running System 7.x.  The main feature missing
from this demo is the ability to save your results.  Otherwise the program is
fully functional.  Additionally, the Propagator network source code generator
utility is not included with this package since it requires saved network
problem files to work.

This document is intended to provide a brief introduction to how to operate
Propagator.  It is not intended to replace the Propagator User's Manual.  In
addition to the information provided here, there are a series of help screens
inside Propagator that can help answer questions that you might have while
running Propagator.



**** Section 2 - Using Propagator Demo

Running an Example Problem

If you are somewhat familiar with backpropagation, you should be able to run a
problem in a matter of minutes.  There are some example problems included with
the demo for you to use while learning to use Propagator.  This demo version
will allow you to use your own files, and the file format is discussed below.
One of the example problems is called the encoder problem.  This problem
attempts to replicate the input values as the output values while passing the
data through a smaller number of hidden nodes.  This requires the network to
learn to encode or compress the data from input layer to hidden layer and to
decode or uncompress again from hidden to output layer.  The following
abbreviated instructions describe how to setup and run the encoder problem.

Start Propagator by double clicking the Gator icon.
     - The control window will appear

To define the network architecture, select the "Architecture" button on the
Control Window.
     - The Network Architecture dialog will appear
     - Use this dialog to define the architecture

Enter the following settings in the dialog:
     - Number of layers = 3
     - Nodes per layer
          - Input layer = 8
          - Hidden layer = 3
          - Output layer = 8
     - Transfer functions
          - Hidden layer = Sigmoid
          - Output layer = Sigmoid
     - Fully Connected
     - Randomize weights
          - Low value = -1.0
          - High value = 1.0
     - Generalized Delta Rule
     - Time-based random seed
     - Press "OK" to close the Network Architecture Dialog.

To define the training parameters, select the "Training Parameters" button on
the Control Window.
     - The Network Training Parameters dialog will appear
     - Use this dialog to define the training parameters

Enter the following settings:
     - Learning rate = 0.5
     - Momentum factor = 0.9
     - Total training cycles = 2000
     - Min training error = 0.0
     - Update interval = 10
     - Random presentation
     - No input noise
     - Press the "Open Training File..." button
          - This will display an Open dialog.  Select the file encoder.trn
               in the encoder directory.
     - Press the "Open Validation File..." button
          - This will display an Open dialog, select the file encoder.val
               in the encoder directory.
     - Press "OK" to close the Training Parameters Dialog.

To run the encoder problem, press the "Train" button from the Control Window.
     - The problem will start running.
     - You can stop the problem manually by pressing "Stop" button.
     - You can open any or all of the graphs from the Graphs menu.

That's all it takes to run a Propagator neural network.  There are many
other features that Propagator provides.  They are explained in the sections
below.  But, first the Propagator file format is explained so you can test
Propagator with you own data.



**** Section 3 - Preparing Your Own Data Files

The Scale utility is included with the Propagator demo.  This program performs
two main functions.  First, it changes any non-numeric delimiter characters to
tabs '\t' and newlines '\n' used by Propagator.  Second, it will scale the
input and target values to ranges specified by you.  This section will explain
the format of Propagator input files training, validation, and testing.  All
the files use the same format.  Data files contain two parts, input and
targets; each input is paired with its target.  An input-target pair is called
a pattern.  Each data file may contain many patterns.
     1. Data files must be in ASCII format.
     2. All inputs and targets must be numbers.
          - floating point and integer are both allowed
          - zero and negative numbers are also allowed
     3. Data files take the following format.
          Pattern 1 inputs         Pattern 1 targets
          Pattern 2 inputs         Pattern 2 targets
          Pattern 3 inputs         Pattern 3 targets
               ...                 ...
          Pattern n inputs         Pattern n targets
     4. Inputs and targets must be separated by one or more white spaces.
          - spaces ' ', tabs '\t', or newlines '\n'
     5. Any number of items can be places on the same line.
          - patterns can be split onto many lines
          - several patterns can be put on one line

Acceptable example for three patterns, each with 2 inputs and 3 targets:
     0.21 9.8  9.1  3    7
     .1   .2   .3   .4   0.5
     .001 .9   100  1    1.1
For the first pattern .21 and 9.8 are the inputs, and 9.1, 3, and 7 are the
targets.

Unacceptable example for three patterns, each with 2 inputs and 3 targets:
     0.21,     9.8, 9.1, 3,   7
     .1,  .2,  .3,  .4,  0.5
     .001,     .9 Last pattern

The problems with this data are:
     - Commas cannot be used as delimiters.  The Scale program can
          remove them.
     - The number of items in the file does not match the network
          architecture.  Since there are 5 items in a patterns (2 + 3)
          there must be a multiple of 5 number of items in the file.
     - The text "Last pattern" cannot appear in the file.  All entries must
          be numbers.

The Scale program makes it much easier to prepare input data for Propagator.
This program performs three basic functions.
1. Scales the input and target data and places it in a file.
2. Sets the delimiters to tabs '\t' between items and newlines '\n' at the end
     of each pattern.
3. Creates a log file detailing the scaling process.

To run Scale utility, double click on the Scale icon.

Enter the following values in the Scale program window:
     - The number of input nodes.
     - Click the Scale Input Patterns check box if you want to scale the
          inputs (otherwise the input values will remain the same, and
          only the delimiters will be changed).
     - The number of target nodes.
     - Click the Scale Target Patterns check box if you want to scale the
          inputs (otherwise the target values will remain the same,
          and only the delimiters will be changed).
     - Select the Select Input File... button.
          - Use this to select your raw input data file.
     - Select the Select Output File... button.
          - Use this to name the output file for the scaling results.

To start scaling, press the Scale button.
     - The process will create a file useable by Propagator.



**** Section 4 - The Control Window and the Dialog Boxes

Control Window

When you start Propagator, by double clicking the Propagator
icon, the control window will be displayed.  The control window is command
central for Propagator.  From this window, you can view all of your current
setting, view the current status of the network, access the architecture
dialog, access the training parameters dialog, access the testing parameters
dialog, view the performance history dialog, start training, stop training and
start testing.  Clicking on the three buttons on the left side of the control
window (Architecture, Training, Testing) open the appropriate dialog box.
These dialog boxes can also be accessed from the network menu.  The three
buttons on the bottom of the screen are used to start and stop training and
testing.  You can start and stop training as often as you want.



Network Architecture Dialog

The architecture dialog is used to design your
neural network.  There are a variety of setting that are available.  Once you
make your setting and start training you cannot access the dialog box again.
In order to change the architecture, you must either start a new problem or
select reinitialize from the network menu.  The following parameters are
defined in the Network Architecture dialog box:

Number of Layers - Propagator can use from two to five layers.

Nodes per Layer - Specifies how many nodes make up each network layer.  You
can have from 1 to 32768 nodes per layer.

Transfer Functions - Functions assigned to each layer except the input layer
which is always linear.  Three functions are available: Linear, Sigmoid, and
Tanh.  Generally, at least one of the layers uses a nonlinear transfer
function (tanh or sigmoid).  When using tanh, data normalization is usually
between -1 and 1.  When using sigmoid, data normalization is between 0 and 1.

Connectivity - Enables selection of fully connected or partially connected
networks.  If a partially connected network is used, a Partial Connection File
must be specified.  This file determines which nodes are connected.  Most
applications use fully connected networks.

Weight Initialization - Used to assign Weight Initialization values.  For the
network to learn, all of the initial weights cannot be the same.  Propagator
enables you to specify low and high range limits for the initial weight
values.  Propagator assigns the weights and bias units random numbers between
these two values.  The valid range for each of the limits is -9999.99 to
9999.99, and the low value must be less than the high value.

Learning Rule - Propagator permits the use of either the Generalized Delta
Rule or the Cumulative Delta Rule.  Both are variations of backpropagation.
The difference between the two rules is that the Generalized Delta Rule
updates the weights after every training pattern and the Cumulative Delta Rule
updates the weights at the end of each cycle.  When using the Generalized
Delta Rule, the training patterns are typically presented to the network in
random order.  To save time, they should be presented sequentially when using
the Cumulative Delta Rule.

Random Seed - There are two options for Random Seed: Time-based or User
Specified.  If Time-based is selected, Propagator uses a random seed based on
the current time.  Time-based seeds differ each time a network is run.  To
replicate past results, specify a random seed for Propagator to use.  The
value of the seed must be between 0 and 2,147,483,647.



Network Training Parameters Dialog

This dialog box is used to set the values of the Network Training Parameters.
These parameters can be changed after training has started.

Learning Rate - Determines the magnitude of the weight change during training.

Generally, values between 0 and 1 are used.  If 0 is selected, the network
will not learn.  If the Learning Rate is too large, the error will oscillate
and the network will not learn.

Momentum Factor - This parameter helps networks learn faster by basing the
current weight changes proportionally on the previous weight changes.
Generally, values between 0 and 1 are used.  If 0 is selected, the Momentum
Factor term is not used.

Training Patterns Order - The training patterns can be presented either in
random order or sequentially in the order of the patterns in the Training
File.  If the Cumulative Delta Rule is used, the order does not matter,
although sequential presentation is recommended to save time.

Total Training Cycles - Determines the maximum training cycles the network can
run.  For example, if Total Training Cycles is set to 100, network training
stops after 100 training cycles.  To train the network an additional 100
cycles, this value must be increased to 200.

Minimum Training Error - Enables the network to stop training when the
specified error value is reached.  The training error is the sum of the
squared differences between the outputs and target values over the entire
cycle divided by the number of training patterns.  This is often referred to
as the Mean Squared Error.  Values in the range of 0.1 are often used.

Update Interval - Determines the behavior of many Propagator functions.  The
valid range for the interval is 1 to 2,147,483,647, and it must be less than
the Total Training Cycles value.  The following items are triggered when the
number of training cycles reaches a multiple of the Update Interval:
1. The validation set is presented to the network,
2. A new line is added to the Performance History area of
     the Control Window, and
3. Any graph currently shown is updated.

Training Patterns Input Noise - Gaussian noise can be added to the training
patterns.  If added, the mean and standard deviation of the noise must be
specified.  Generally, a mean of 0 and a small standard deviation such as 0.5
are used.  Adding noise to the training patterns can enable the network to
learn a more general solution to a problem.

Training File - Determines the data file used for training the network.  The
Open Training File... button displays the Open File dialog that enables you to
select a Training File.  When the Train button is pressed in the Control
Window, Propagator verifies that the file contains the proper number of items.

The Training File can be changed after training has started.

Validation File - Determines the data file used for in the validation passes
of the network.  The Open Validation File... button displays the Open File
dialog that enables you to select a Validation File.  When the Train button is
pressed in the Control Window, Propagator verifies that the file contains the
proper number of items.  The Validation File can be changed after training has
started.  However, once training with validation is started, the Validation
File cannot be deleted.  Likewise, if training without validation is started,
you cannot add a Validation File.  The Validation File is optional, and does
not need to be used to train a network.



Network Testing Parameters Dialog

This dialog box enables you to select a file to use in testing a network and
to add noise to the inputs of this Test File.  After all entries in this
dialog are complete and it is closed, the test is run by selecting the Test
button on the Control Window.  The results of the test are displayed in the
Performance History area of the Control Window.  You can optionally request an
Output vs. Target File from your test by selecting the Write Output vs. Target
File When Testing option in the Preferences dialog.

Testing Patterns Input Noise - Gaussian noise can be added to the testing
patterns.  If added, The mean and standard deviation of the noise must be
specified.  Generally, a mean of 0 and a small standard deviation such as 0.5
are used.  Adding noise to the testing patterns can enable you to determine
more easily how well your network will generalize.

Testing File - Determines the data file used for testing the network.  The
Open Testing File... button displays the Open File dialog that enables you to
select a Testing File.  You can select any file, including a Training or
Validation File, that has the proper number of elements.



**** Section 5 - The Graphs

Propagator displays three graphs that can help you determine the progress of
network training.  Each of these graphs can be opened and closed while
Propagator is running, and each is fully resizable.  A brief description of
each graph follows.



Error vs. Cycle Graph

The Error vs. Cycle graph displays the mean squared error of the training and
validation sets as training progresses.  If a validation set is not used,
only the training set graph is displayed.  The mean squared error is plotted
on the y-axis and the cycle number is plotted on the x-axis.  This graph is
updated according to the Update Interval specified in the Network Training
Parameters dialog.

Rescale - You can rescale the graph by clicking and dragging the mouse to
create a rectangular region inside the active graph region.  When you release
the mouse button, the graph rescales showing the selected area of the graph in
greater detail.  You can zoom in as many times as you like.  To zoom out
select the Zoom Out button.



Error vs. Output Unit Graph

The Error vs. Output Unit graph simultaneously displays the mean squared error
for each output node in the network.  The sum of all of the error on this
graph equals the training error value on the Error vs. Cycle graph.  The mean
squared error is plotted on the y-axis and the output node number is plotted
on the x-axis.  When all of the error values are within twenty percent of
zero, the graph is automatically rescaled for better viewing.



Output Error vs. Target Graph

The Output Error vs. Target graph shows the relationship of an individual
output node's errors to the target values for all patterns in a training
cycle.  The errors, which are the output minus target values, are plotted on
the y-axis while the target values are plotted on the x-axis.  As performance
improves the values should tend to approach the zero error line drawn in the
middle of the graph.  The slider at the bottom of the graph allows you to
select the output node for which the error values are displayed.  When all of
the error values are within twenty percent of zero, the graph is automatically
rescaled for better viewing.



**** Section 6 - Other Features

Propagator provides several other function all of which are not useable in the
demo.  In particular, the Output files listed in the Preference Dialog are not
created in the demo.  These files are created when the problem file is saved,
and since you cannot save with the demo version these files are not created.
The source code generator is also absent from the demo version.



Jog Weights

Jog Weights dialog is used to specify a range of values by which the weights
and bias units in an existing network can be altered randomly.  This technique
is most often employed when the network is stuck in a local minimum.

Propagator randomly selects a number between the high and low range limits and
adds it to each weight and bias unit in the network.  Generally a small range
of values such as -0.1 to 0.1 is used.  Each weight and bias unit has a
different random number within the desired range added to it.  The random
numbers are drawn from a uniform distribution.



Preferences

Since you cannot save your trained networks in the demo version most of the
Preferences dialog does not apply to the demo version.  However, the Training
Speed/Responsiveness slider lets you set the comprise between the speed the
network trains and how responsive the program is to user interactions.



Reinitialize

Reinitialize feature is used to start a problem over again without having to
reenter the various parameters in the dialog boxes.

Reinitialize performs the following functions:
1.  Sets the current cycle to 0,
2.  Initializes the weights,
3.  Clears the Performance History area of the Control Window, and
4.  Allows access to the Network Architecture dialog.



**** Section 7 - Ordering Information

To order Propagator please contact:

     ARD Corporation
     9151 Rumsey Road
     Columbia, MD 21045
     Phone     [phone] or 1-800/969-2731
     Fax       [phone]
     Email     [e-mail]

The pricing for Propagator is:
      PC Windows 3.x version                       $199
      Macintosh System 7.x version                 $199
      Sun Solaris 1.x and Solaris 2.x version      $499
We accept VISA and Master Card.  Educational discounts, site licenses, and
free technical support available.  We also provide a money back guarantee.

Finally, ARD's Software Systems Group provides complete software consulting
services.  We have written complex applications for many industries since
1980.